Driver assistance system for vehicle

ABSTRACT

A driver assistance system for a vehicle includes an imager disposed at the vehicle and viewing exterior the vehicle. A control includes an image processor and is operable to process captured image data for at least a first application and a second application and to adjust the image sensor to at least two settings and to process captured image data via at least two processing techniques. The control is operable to synchronize the at least two settings of the image sensor and the at least two processing techniques to extract respective information from the captured image data for the first and second applications. The control adjusts a setting of the image sensor to capture image data suitable for the first application or the second application. The image processor processes captured image data via a processing technique suitable for the first application or the second application.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 13/936,701, filed Jul. 8, 2013, now U.S. Pat. No. 8,977,008,which is a continuation of U.S. patent application Ser. No. 13/481,100,filed May 25, 2012, now U.S. Pat. No. 8,483,439, which is a continuationof U.S. patent application Ser. No. 13/017,353, filed Jan. 31, 2011, nowU.S. Pat. No. 8,189,871, which is a continuation of U.S. patentapplication Ser. No. 11/239,980, filed Sep. 30, 2005, now U.S. Pat. No.7,881,496, which claims the benefit of U.S. provisional application,Ser. No. 60/628,709, filed Nov. 17, 2004; and Ser. No. 60/614,644, filedSep. 30, 2004, which are all hereby incorporated herein by reference intheir entireties.

FIELD OF THE INVENTION

The present invention relates to imaging systems for vehicles and, moreparticularly, to reverse aid imaging systems with a rearward facingimaging device or camera and a display. However, aspects of the presentinvention are equally suitable for other vehicle imaging systems, suchas side object detection systems, and forward facing imaging systems andthe like.

BACKGROUND OF THE INVENTION

The advent of low cost, reliable imaging devices, based on a variety ofsilicon technologies, and in particular CMOS technology, combined withan improved cost/performance ratio for displays capable of meetingautomotive specifications, and an increasing application rate of videomonitor displays for automotive navigation systems or as part of thedriver interface to a wide variety of vehicle systems, has lead to anincreasing use of cameras or imaging sensors designed to give the drivera view of those areas around the vehicle which are not in the normaldirect field of view of the driver, typically referred to as “blindspots”. These areas include the region close to the front of thevehicle, typically obscured by the forward structure of the vehicle, theregion along the passenger side of the vehicle, the region along thedriver side of the vehicle rearward of the driver, and the areaimmediately rearward of the vehicle which cannot be seen directly orindirectly through the rear view mirror system. The camera or imagingsensor may capture an image of the rearward (or sideward or other blindspot area) field of view, and the image may be displayed to the driverof the vehicle to assist the driver in backing up or reversing orotherwise driving or maneuvering the vehicle. The use of electroniccameras in these applications significantly increases the driver'sknowledge of the space immediately surrounding the vehicle, which may beof importance prior to and during low speed maneuvers, and thuscontributes to the safe completion of such maneuvers. However, in orderto provide user controls or user inputs to allow the driver or user tocontrol various functions of the camera at the rear or side or front ofthe vehicle, additional wiring or connections or communication links maybe needed between a control and user input in the vehicle and the cameraor imaging device at the rear or side or front of the vehicle.

It is known to provide a headlamp control system having an imagingsensor positioned on a vehicle and having a forward field of view. Thesystem may detect headlamps and taillights in the images captured by theimaging sensor. It is also known to provide a lane departure warningsystem that includes an imaging sensor positioned on a vehicle andhaving a forward field of view. The lane departure warning systemdetects lane markers and the like along the road surface in front of thevehicle and determines when the vehicle is drifting out of the lane, andmay provide an alert to the driver of the vehicle when such drifting isdetected. Such systems typically are separate systems with different,independently operable controls and image processors and imagingsensors.

SUMMARY OF THE INVENTION

The present invention provides an imaging and display system or visionsystem for a vehicle that captures images of a scene occurringexteriorly of the vehicle, such as rearward of the vehicle, and displaysthe captured images at a display device in the vehicle. The imaging anddisplay or vision system includes an imaging device or camera that ispositioned at the vehicle with an exterior field of view (such as arearward field of view) for capturing images of the exterior scene. Thedisplay device is positioned within the vehicle and remote from thecamera and is operable to display the captured images in the vehiclewhere they are readily viewable by the driver or occupant of thevehicle. The vision system includes one or more user inputs at thedisplay device and is operable to adjust or control at least onefunction or mode or feature of the camera in response to actuation ofthe user input or inputs. The vision system is operable to communicateimaging signals from the imaging device to the display device and tocommunicate camera control signals from the display device to theimaging device along common connections or wiring or communication linksbetween the imaging device and the display device.

According to an aspect of the present invention, a vision system for avehicle includes an imaging device having an imaging sensor, a cameramicrocontroller, a display device having a display element, a displaymicrocontroller, and at least one user input selectively actuatable by auser. The user input is selectively actuatable by a user. The imagingdevice communicates an image signal to the display device via acommunication link. The display microcontroller affects the image signalin response to the at least one user input. The camera microcontrollermonitors the image signal on the communication link and adjusts afunction of the imaging device in response to a detection of theaffected image signal.

The imaging sensor may have a field of view exteriorly of the vehicle,such as rearwardly of the vehicle, for capturing an image of the sceneoccurring exteriorly of the vehicle. The vision system may includecontrol circuitry at or near or associated with an interior rearviewmirror assembly of the vehicle or a windshield electronics module oraccessory module of the vehicle.

The display microcontroller may affect the image signal by disabling avideo termination at the display device, and the imaging device may stopcommunicating the image signal when the video termination is disabled.The display microcontroller may selectively apply at least two voltagelevels across the communication link in response to at least two userinputs, and the camera microcontroller may adjust a function of theimaging device in response to a respective one of the voltage levels.

The imaging device may include a video encoder that communicates theimage signal to the display device via the communication link. Thecommunication link may comprise a video plus and a video return wire.

According to another aspect of the present invention, a vision systemfor a vehicle includes an imaging device having an imaging sensor, acamera microcontroller, a display device having a display element, adisplay microcontroller, and at least one user input selectivelyactuatable by a user. The imaging device communicates an image signal tothe display device via a communication link, and the displaymicrocontroller communicates a control signal to the imaging device viathe communication link in response to the user input. The cameramicrocontroller receives the control signal and adjusts a function ofthe imaging device in response to the control signal. The image signaland the control signal utilize a common link between the imaging deviceand the display device.

According to another aspect of the present invention, an imaging anddisplay system or vision system may be operable in combination with orin conjunction with an auxiliary or triggering or initial sensing deviceor system or distance sensing/measuring/determining system. Theauxiliary sensing device or system may provide additional sensing areasto cover blind spots that may not be encompassed by the imaging sensoror camera. Optionally, the auxiliary sensing system, such as ultrasonicsensors, radar, lidar, and the like, may detect an object exteriorly ofthe vehicle, and may determine or measure or detect the distance to theobject, whereby the imaging and display system may be adjusted orcontrolled to provide enhanced imaging of the exterior scene or enhancedprocessing of the captured images or enhanced displaying of the imagesof the exterior scene and the detected object, in response to such anobject detection by the auxiliary sensing system. For example, theimaging and display system may adjust a camera setting to provideenhanced imaging of the detected object, or may adjust the processor orcontrol to provide enhanced processing of a portion of the image datathat corresponds to the region of the scene at which the object wasdetected, or may adjust the display to highlight or enhance thedisplayed images of the detected object or of the region of the displaythat corresponds to the region of the exterior scene at which the objectwas detected.

Therefore, the present invention provides an imaging and display systemor vision system that includes an imaging device and a display deviceconnected via video connections or communication links. The imagingdevice includes user inputs and the imaging and display system isoperable to selectively control or adjust the imaging device in responseto the user inputs. The imaging and display system or vision system isoperable to communicate imaging signals from the imaging device to thedisplay device and to communicate camera control signals from thedisplay device to the imaging device along common connections or wiringor communication links between the imaging device and the displaydevice. The imaging and display system or vision system thus may providevarious optional features or functions or modes without having to changeor add wiring or connections between the imaging device and the displaydevice.

The present invention also provides an imaging system or vision systemfor a vehicle that is operable to capture images of a scene occurringexteriorly of the vehicle, such as forward of the vehicle, and thatcaptures and/or processes selective image data differently than otherimage data to extract and analyze the desired data for differentapplications or systems or accessories. The imaging system includes animaging device or camera that is positioned at the vehicle with anexterior field of view (such as a forward field of view) for capturingimages of the exterior scene. The imaging system may be operable toselectively or intermittently capture and/or process some image data ina manner suitable for a lane detection and/or to capture and/or processother image data in a manner suitable for headlamp detection and/or tocapture and/or process other image data in a manner suitable for ambientlight detection and/or other functions, systems or features or the like.

According to another aspect of the present invention, an imaging systemfor a vehicle includes an image sensor and a control. The image sensoris positioned at a vehicle and has an exterior field of view. The imagesensor is operable to capture images of the exterior field of view andto generate image data. The control is operable to adjust the imagesensor to at least two settings, and to process the image data via atleast two processing techniques. The control is operable to synchronizethe image sensor settings and the processing techniques to extractrespective or appropriate information from the captured images for atleast two applications of the imaging system.

The image sensor may have a field of view forwardly of the vehicle forcapturing an image of the scene occurring forwardly of the vehicle.

The control may adjust the image sensor setting to capture image datasuitable for a particular application, and may process those capturedimages via a processing technique suitable for the particularapplication. The control may adjust the image sensor to other settingsand may correspondingly process those captured images via otherprocessing techniques depending on the desired or appropriate orparticular or respective application or function of the imaging system.For example, the control may selectively or intermittently oroccasionally adjust the image sensor setting and processing technique toextract information from the captured image data for a headlamp controland/or a lane departure warning system and/or a rain sensor and/or anavigational system and/or an ambient light sensor and/or a collisionavoidance system and/or a driving separation indicator and/or a backupaid and/or an object detection system and/or the like.

Therefore, the present invention also provides an imaging system orvision system that includes an imaging sensor and a control that areoperable to provide multiple functions or control of multipleaccessories or the like via selective capturing of image data andprocessing of the captured image data. The control is thus operable toselectively capture images and process image data to provide multiplefunctions or applications with a common image sensor and imageprocessor. The imaging system may synchronize the settings of the imagesensor with the processing techniques applied to the image data by theimage processor, in order to perform and/or optimize two or morefunctions of the imaging system. The present invention thus provides amulti-tasking capability to an image sensor and image processor whileproviding enhanced capturing of the images and processing of the imagedata for the desired functions or applications.

These and other objects, advantages, purposes and features of thepresent invention will become apparent upon review of the followingspecification in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a rear perspective view of a vehicle having an imaging anddisplay system thereon in accordance with the present invention;

FIG. 2 is a plan view of the vehicle of FIG. 1;

FIG. 3 is a block diagram of a camera and display device, showing theconnections or wiring typically used to connect the camera to thedisplay device;

FIG. 4 is a block diagram of an imaging and display system in accordancewith the present invention;

FIG. 5 is a plan view of a vehicle incorporating an imaging system inaccordance with the present invention;

FIG. 6 is a side elevation of a portion of a vehicle embodying animaging system in accordance with the present invention;

FIG. 7 is a block diagram of an imaging system having an imager and acontrol;

FIG. 8 is a block diagram of an imaging system in accordance with thepresent invention, showing different exemplary applications of thecontrol; and

FIG. 9 is another block diagram of the imaging system of the presentinvention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring now to the drawings and the illustrative embodiments depictedtherein, an image capture system or vision system or imaging and displaysystem 10 is positioned at an exterior portion 12 a of a vehicle 12,such as at a rearward portion 12 a of the vehicle 12, and is operable tocapture an image of a scene occurring exteriorly of the vehicle, such asrearwardly of the vehicle, and to display the image at a display deviceor display system 14 of the vehicle which is viewable by a driver of thevehicle (FIGS. 1 and 2). Vision system 10 includes display device orsystem 14 and an imaging system or imaging device 16, which includes animager or image capture device or camera 22 that is directed exteriorlyof the vehicle and has an exterior field of view 17 which preferably atleast partially encompasses a “blind spot” area exteriorly of thevehicle. The images or frames captured by imaging system 16 aredisplayed at a display element 28 of display system 14 to assist thedriver in viewing the blind spot areas. The display system 14 includesone or more user inputs 18 (FIG. 4) that are actuatable by a user of thesystem to control or adjust a function of the imaging system 16, whichis remote from the display system 14. The vision system 10 is operableto communicate imaging signals from the imaging device or system to thedisplay device or system and control signals from the display device orsystem to the imaging device or system along common connections orwiring.

Imaging system or imaging device 16 may be positioned at the exteriorportion of the vehicle and directed generally exteriorly of the vehiclefor capturing images of the exterior scene to assist the driver inmaneuvering or driving the vehicle. Vision system 10 and/or imagingsystem 16 may utilize principles of other vehicle vision or imagingsystems, such as a vision or imaging system of the types disclosed inU.S. Pat. Nos. 6,757,109; 6,717,610; 6,396,397; 6,201,642; 5,550,677;5,670,935; 5,877,897; 5,796,094; 6,097,023; and 6,498,620, and U.S.patent application Ser. No. 09/441,341, filed Nov. 16, 1999, now U.S.Pat. No. 7,339,149; and/or U.S. provisional application, Ser. No.60/614,644, filed Sep. 30, 2004; and Ser. No. 60/618,686, filed Oct. 14,2004, which are hereby incorporated herein by reference. In theillustrated embodiment, the vision system is operable to capture anddisplay images of the rearward area immediately behind the vehicle, soas to assist the driver in backing up or otherwise driving ormaneuvering the vehicle rearwardly. However, the vision system may beoperable to capture and display images of other areas exteriorly of thevehicle to provide images of other blind spot areas around or near thevehicle, without affecting the scope of the present invention. Thevision system thus may be operable to captures images of the sceneimmediately rearward of the vehicle to assist the driver of the vehiclein backing up or maneuvering the vehicle in reverse. The backup assistsystem may be operable in response to the reverse gear of the vehiclebeing selected.

Preferably, the display element or display device may be located at anupper windshield area of the vehicle so as to be readily viewable by thedriver of the vehicle. The display element may be located at or near theinterior rearview mirror assembly of the vehicle or may be associatedwith the interior rearview mirror assembly of the vehicle. Optionally,the display element may be located at or near an accessory module or podor windshield electronics module of the vehicle or may be associatedwith the accessory module or pod or windshield electronics module of thevehicle.

The imaging system may process the captured image data to detect objectsor the like in the exterior scene. Such data processing may utilizeaspects of the image processing techniques described in U.S. Pat. Nos.6,353,392 and 6,313,454; and/or U.S. patent application Ser. No.10/427,051, filed Apr. 30, 2003, now U.S. Pat. No. 7,038,577; and/orU.S. provisional application Ser. No. 60/638,687, filed Dec. 23, 2004,which are hereby incorporated herein by reference. The display device orelement may display the image of the scene or may display the detectedobjects and/or may display graphic overlays or the like (such asdistance measurements, icons, text or the like) to enhance the displayfor viewing by the driver or occupant of the vehicle. The vision systemmay be selectively operable to provide different functions or modes orfeatures as selected by the user via one or more user inputs, asdiscussed below.

The imaging system 16 of vision system 10 thus may be operable in two ormore modes and/or may provide various functions or features that areselectable by a user of the system (such as the driver of the vehicle)via one or more user inputs at or near or associated with the displaysystem. For example, the imaging system may be selectively operable in acolor mode or a black and white mode or a daytime or nighttime mode, ormay be selectively operable to zoom in on a detected object or to panacross a scene or to adjust the gain, exposure and/or white balance inthe captured images, in response to actuation of a respective user inputby a user within the vehicle. Optionally, the imaging system may beselectively operable to process the images and display the images of theexterior scene in an outline form, where only the outlines of detectedobjects are shown at the display element, such as in a black and whiteline format. The image thus may be manipulated to provide an outlineform and thus may be suitable for display on a low cost display element,such as a dot matrix display or the like, such as the type described inU.S. patent application Ser. No. 10/418,486, filed Apr. 18, 2003, nowU.S. Pat. No. 7,005,974, which is hereby incorporated herein byreference. Also, or otherwise, the vision system may provide user inputsthat enable a user to selectively actuate an overlay image to overlayimages, such as distance measurements, text, icons, an imagerepresentation of the back of the vehicle, or other overlay images oricons, onto or over the image of the exterior scene, in order to enhancethe display of the exterior scene for the user.

As shown in FIG. 3, and as is typical for various known imaging devicesand vision systems, there are five connections or wires or leads orcommunication links 20 between the display system 14 and the imagingsystem 16. The connections or links include a camera power connection 20a, a ground connection 20 b, a video + connection 20 c, a video returnconnection 20 d and a shield 20 e. In known reverse aid systems,providing a control of the camera, such as to activate a digital zoom orpan mode for the camera (or to otherwise control or adjust a camerasetting or mode), from the display area (and a user input at or near orassociated with the display device) requires additional wiring for thesecondary communication or communications between the user input of thedisplay device and the camera and, thus, is a costly option for theautomotive market and makes retrofitting a reverse aid system on avehicle very difficult and costly. Controlling or adjusting or actuatingthese camera functions or modes or settings from the display area withinthe vehicle increases the overall system level performance and addsadditional functionality to the system.

As shown in FIGS. 3 and 4, vision system 10 includes the communicationlinks 20 (only the video + connection 20 c and the video returnconnection 20 d are separately shown in FIG. 4 for purposes of clarity)between the imaging system 16 and display system 14. As shown in FIG. 4,the imaging system 16 of vision system 10 includes an imaging sensor orimager 22, a camera microcontroller 24 and a video encoder 26. Theimager or imaging array sensor 22 captures image data indicative of theexterior scene and generates a data signal 22 a and outputs the datasignal to the video encoder 26, which encodes the data and communicatesan encoded video signal 26 a to the display system 14 via the video +and video return connections 20 c, 20 d. Under normal operation, thecamera typically outputs an NTSC video signal on the video + and video −or return connections 20 c, 20 d to the display system 14. This istypically done at about a 30 hertz frame rate. The cameramicrocontroller 24 controls the imager 22 and the video encoder 26, andmonitors (as shown via monitoring link 24 a in FIG. 4) the encoded videosignal 26 a from the encoder 26 to the display system 14. When a changein the signal 26 a is detected by the camera microcontroller, the cameramicrocontroller 24 may adjust or control the imager 22 and/or videoencoder 26 to set the imaging device or system to a different mode orfunction, as discussed below.

Display system or display device 14 includes a display element 28 and adisplay microcontroller 34. The display system 14 receives the encodedvideo signal 26 a from the imaging system 16 via the video + and videoreturn connections 20 c, 20 d, and displays the captured image on thedisplay element 28, such as an LCD panel or the like (or other types ofdisplays or display elements as discussed below). As shown in FIG. 4,display system 14 includes a video buffer and termination 30, a videodecoder 32 and the display element 28. The video buffer and termination30 receives the video signal 26 a and passes the signal to the videodecoder 32, which decodes the encoded video signal and communicates animage signal or RGB signal 32 a to the display element 28. The displayelement 28 displays the captured image within the vehicle where it isreadily viewable by a driver or occupant of the vehicle.

Display system 14 also includes display microcontroller 34 that isoperable to receive a user input signal 18 a from one or more userinputs 18 and to adjust or control the display element 28 and/or toenable or disable the video termination 30 in response to the user inputsignal 18 a. When the video termination is disabled, the cameramicrocontroller 24 detects the change in the signal along the video +and video return connections 20 c, 20 d, and may adjust or control theimager 22 and/or video encoder 26 in response to such detection.

During normal operation of the imaging system, the imaging system 16 mayoutput the NTSC video signal (or any other video signal protocol whetheranalog or digital) on the video + and video − or return connections tothe display system 14. The camera microcontroller 24 may continuously orsubstantially continuously monitor the NTSC video output signal 26 a fora normal output that is about one volt peak into a 75 ohm load (providedat the video buffer and termination 30). When an appropriate user input[such as a button or switch or the like at or near or associated withthe display element or a voice command or a consequent of a user action(such as selection of a reverse gear of the vehicle or the like) or avehicle action or condition or characteristic or status (such as inresponse to the vehicle reaching a particular or threshold forward orreverse vehicle speed or the vehicle encountering a braking condition orthe like) or other input or condition or characteristic or status or thelike] is activated by a user action (such as a user input forselectively adjusting a camera function or mode, such as a digital zoomfunction or pan function or and adjustment of a gain or exposure orwhite balance function or a selection of a color mode or a black andwhite mode or a night vision mode or the like), the displaymicrocontroller 34 disables the video termination 30 at the displaysystem 14 via a signal 34 a. The disabling of the video termination 30at the display system 14 causes the video encoder 26 of the imagingsystem 16 to stop outputting the NTSC video signal 26 a. The cameramicrocontroller 24 senses the change in the output signal or voltagealong the connections 20 c, 20 d, and may control or adjust the imagerto a different or new camera setting in response to such a detectedchange in signal output/voltage, or may control or adjust the videoencoder if appropriate in response to such a detected change in thevideo signal along the connections 20 c, 20 d.

Optionally, when multiple functions or modes are desired, the displaymicrocontroller 34 may disable the video termination and apply aparticular voltage across the connections 20 c, 20 d (via signal 34 a)that corresponds to the particular function or adjustment selected bythe user at the user inputs, and the camera microcontroller 24 may beoperable to determine the voltage applied on the connections and toadjust or control the imager and/or encoder accordingly. The cameramicrocontroller 24 may be preset or programmed to recognize theparticular voltage across the connections and to control or adjust oractivate the respective function or feature or mode in response to suchdetection and recognition. The camera adjustments or settings (such asactivation of a digital zoom feature, a panning feature, a gain orexposure or white balance function or a black and white mode or nightvision mode or the like) thus may be predetermined and based on the DCvoltage level applied across the connections 20 c, 20 d by the displaymicrocontroller 34 and sensed across the connections 20 c, 20 d by thecamera microcontroller 24.

After the camera adjustment is made by the camera microcontroller, thedisplay microcontroller may enable the video termination and remove theapplied DC voltage if applicable, so that the encoder 26 will againcommunicate the video signal 26 a to the display system 14. Optionally,the display microcontroller 34 may wait a predetermined amount of timebefore enabling the video termination again and removing the DC voltagelevel. This delay allows the camera microcontroller enough time to sensethat the termination has been removed, that the new programmed settingshave been uploaded to the imager, and that the settings have takenaffect. Once the delay is completed, the video termination is enabled atthe display, and the camera microcontroller and video encoder resume theNTSC signal to the display system to communicate the capture image datato the display system.

This process may be repeated for invoking and removal of special cameramodes or functions. For example, for different user inputs andassociated functions, the display microcontroller may disable the videotermination and may apply a different voltage to the connections 20 c,20 d. The camera microcontroller may then determine the voltage in theconnections 20 c, 20 d and may adjust the imager and/or video encoderappropriately and in response to the detected voltage. The number ofspecial modes of operation or functions that are desired to control oractivate/deactivate at the display will dictate the number of voltagesthat are to be resolved by the camera microcontroller at the camera inorder to activate or control or adjust the multiple camera settings fromthe display inputs. For example, if four different functions or modesand associated user inputs were desired, the display microcontroller mayapply different voltages, such as at ¼ volt increments or the like, forthe different inputs and functions, and the camera microcontroller maybe operable to discern or resolve the different voltages and control oradjust the imager and/or video decoder according to the detectedvoltage.

The vision system of the present invention thus provides user selectablecontrol or adjustment of an exterior facing camera via user inputs at aninterior display of the vehicle, without requiring additional wiring forthe connection between the inputs and the camera. The present inventionthus is highly suitable for retrofitting imaging and display systems orvision systems and/or selecting an appropriate or desired camera and/ordisplay device or system that provides the desired features withoutconcerns as to the wiring or connections between the camera and imagingsystem and the display element and display system. A user may select anoptional camera that has desired features and may install the camera atthe vehicle and connect to the existing wiring, which may be the basewiring for a base camera that may not have the features of the newcamera. The present invention thus provides for upgrading of a cameraand/or display and/or vision system either as an option installed at thevehicle assembly plant, or as an aftermarket device, without the costand difficulties associated with rewiring or wiring new connectors orlinks between a user input or inputs and the camera. Different optionalcameras and display systems and imaging systems thus be selected by anowner of a vehicle and may be readily installed in the vehicle withouthaving to re-wire the vehicle or provide complicated communication linksbetween the camera and the display device or system. The presentinvention thus allows for selection of a camera and display device orsystem having the desired features or content, without the costs anddifficulties of implementing the high content cameras, since anyselected camera and display device or system may utilize the sameconnections or communication links.

Optionally, the vision system may include or may be associated with anultrasonic or radar device that determines the distance from the rear ofthe vehicle to a detected object. Optionally, other means fordetermining the distance to a detected object may be implemented, suchas the means described in U.S. patent application Ser. No. 10/427,051,filed Apr. 30, 2003, now U.S. Pat. No. 7,038,577; and/or Ser. No.11/105,757, filed Apr. 14, 2005, now U.S. Pat. No. 7,526,103; and/orU.S. provisional application Ser. No. 60/607,963, filed Sep. 8, 2004,which are hereby incorporated herein by reference. The vision system maydetect the distance to objects in various zones or regions behind thevehicle and may provide a graphic or video overlay of the displayedimage to indicate to the driver the distance to one or more detectedobjects. The overlay may be enhanced, such as by flashing or the like,to enhance the viewability of the overlay, such as when a hazardouscondition is encountered, such as when the vehicle is within a thresholddistance to a detected object rearward of the vehicle and in thevehicle's path of travel. The graphic overlay may be initiated oractivated in response to the vehicle being shifted into the reverse gearor may be selectively activated via a user input or the like, withoutaffecting the scope of the present invention.

Optionally, the imaging system and/or camera and/or display system maybe operable in conjunction with or in combination with a distancesensing/measuring/determining system, such as an ultrasonic sensingsystem or the like. For example, and with reference to FIG. 5, animaging and display system or vision system 10′ may include an imagingsensor or camera or imaging system 16′, which may be generally centrallypositioned at the rear portion of the vehicle 12′, and one or moresecondary or auxiliary sensing devices, such as ultrasonic sensors 36,which may be also positioned at the rear portion of the vehicle. In theillustrated embodiment, an ultrasonic sensor 36 may be positioned at ornear each rear corner region 12 b′ of the vehicle 12′. Typically, animaging sensor or camera may provide a wide angle field of view (showngenerally at 38 in FIG. 5) rearward of the vehicle that is approximatelya 120 degree field of view. Such a field of view may not fully encompassthe regions immediately rearward of the vehicle and at or toward theside corners of the vehicle, whereby blind spots may still exist inthose regions. In order to provide enhanced object detection of objectsthat may be positioned at those regions, such as when the vehicle isbacking up, the ultrasonic sensors 36 may be operable to sense or detectobjects in those regions. The imaging and display system may thendisplay an indication at the display device or display system 14′ of adetected object in response to such detection by the ultrasonic sensors.For example, the display may provide an iconistic display of the objectsuperimposed upon the video image of the rearward scene as captured bythe rearward facing camera, or may provide a flashing of the display atthe image on the screen and in the region or regions of the detectedobject, and/or may provide a graphical overlay at the regions of thedisplay that represent the detected object to indicate that an objecthas been detected in one of the rear corner regions or elsewhererearward of the vehicle.

Optionally, an ultrasonic sensing system may include a plurality ofsensors, such as three or more ultrasonic sensors, positioned and spacedacross a rear portion of the vehicle, such as along a rear fender of thevehicle, or along a rear lift gate of the vehicle, or along a rearspoiler of the vehicle, or along a light bar associated with the licenseplate of the vehicle or the like. For example, and as shown in FIG. 5,the sensors may include two side or corner sensors 36 and one or moresensors 40 (such as two sensors 40 in FIG. 5) interspaced across therear portion of the vehicle and between the corner sensors 36.Preferably, a light bar is provided at the license plate of the vehicleand includes the imaging sensor or camera, the ultrasonic sensors, andpreferably a light for illuminating the license plate of the vehicle,such that the imaging sensor and ultrasonic sensors may be provided as amodule at the rear of the vehicle.

The ultrasonic sensors 36, 40 may provide adjacent and overlappingsensing regions to provide a sensing range that encompassessubstantially the entire region behind the vehicle and about eight feetor thereabouts rearward of the vehicle. The imaging sensor or camera 16′may capture images of the rearward exterior scene that is encompassed bythe field of view of the camera, and the display may display images ofthe scene for viewing by the driver of the vehicle when the vehicle isbacking up. The display may be adjusted or modified or enhanced in oneor more of the regions associated with one or more of the ultrasonicsensors in response to a detection of an object by one or more of theultrasonic sensors.

For example, the display may be divided into four zones, each zone ofthe display corresponding to a zone or region covered by a respectiveultrasonic sensor. If that ultrasonic sensor detects an object, then thecorresponding zone of the display may be adjusted or enhanced toindicate to the driver of the vehicle that an object is detected in aparticular zone or region rearward of the vehicle. For example, one ormore of the zones in the display may flash or modulate or may changeintensity to indicate that an object has been detected in the region orzone rearward of the vehicle that corresponds to the enhanced orflashing or modulating display zone. Optionally, the display may providea graphic overlay, such as a measurement bar or the like, to show adistance to a detected object.

Optionally, the ultrasonic sensor or auxiliary sensing device mayprovide a triggering or initial sensing function, and the imaging systemmay adjust a camera setting or characteristic in response to a detectionof an object by one or more of the ultrasonic sensors. For example, thecamera may be controlled or adjusted to electronically zoom to theappropriate zone or region of the captured image in response to adetection of an object by a respective ultrasonic sensor. Because thecaptured image may have distortion due to the wide angle lens or opticof the imaging system and camera, it may be desirable to adjust theimaging sensor or camera to provide enhanced imaging of an objectdetected in the field of view. The camera thus may be zoomed in on theappropriate zone or region to provide enhanced imaging of that zone orregion and to provide reduced distortion of that zone or region and,thus, enhanced imaging and reduced distortion of the detected objectwhen it is displayed on the display. The imaging and display system mayoptionally provide a dynamic overlay at the display to highlight thedetected object in the display, and may display the distance (which maybe detected or determined in response to the ultrasonic sensing devices)to the object.

Optionally, the imaging system may process particular zones or regionsof the image data corresponding to the exterior scene more than otherzones or regions, such as by utilizing aspects of the object detectionsystem described in U.S. patent application Ser. No. 10/427,051, filedApr. 30, 2003, now U.S. Pat. No. 7,038,577; and/or Ser. No. 11/105,757,filed Apr. 14, 2005, now U.S. Pat. No. 7,526,103; and/or U.S.provisional application Ser. No. 60/607,963, filed Sep. 8, 2004, whichare hereby incorporated herein by reference. Optionally, the imagingsystem may process particular zones or regions in response to adetection of an object by one or more of the ultrasonic sensors. Forexample, if one of the ultrasonic sensors initially detects an object atthe left rear corner of the vehicle, the control may process the imagedata that corresponds to that area or region, or may enhance theprocessing of the image data that corresponds to that area or region, inorder to identify or classify or detect the detected object in that areaor region. The imaging and display system may then highlight or enhancethe display at the detected object, such as by a graphic overlay at thedisplay or an adjustment or modulation of the display in the region ofthe detected object. The ultrasonic sensors thus may function to detectan object and to direct the image processor to process the image datathat corresponds to the area in which the object was detected. The imageprocessor thus may process the image data for the regions that may havean object therein, and may provide reduced processing of other regions,and thus may provide efficient and enhanced processing of the capturedimages.

Optionally, the display may be actuated in response to a detection of anobject behind the vehicle when the vehicle is backing up or in reverse.For example, when a driver shifts the vehicle into reverse, theultrasonic sensors at the rear portion of the vehicle may be activatedto sense the area rearward of the vehicle. The imaging and displaysystem may monitor the ultrasonic sensors or may receive an input fromthe ultrasonic sensors or system, and may initially be in a non-activeor non-display mode, where the display is deactivated. When an objectrearward of the vehicle is detected by the ultrasonic sensors, thedetection by the ultrasonic sensor or sensors may trigger the imagingand display system to be activated or set to an active mode, whereby theimaging sensor may begin capturing images of the rearward scene and thedisplay may begin displaying images of the rearward scene (which mayinclude enhanced imaging or processing or display or highlighting of thedetected object such as discussed above). The activation of the displaymay provide an alert function to the driver of the vehicle that anobject is detected behind the vehicle and/or in the path of the vehicle,so as to draw the driver's attention to the activated display and to theobject being displayed in the display.

Optionally, the imaging and display system and ultrasonic sensing systemmay be implemented in conjunction with a movable video display, such asa slide out or extendable/retractable display screen that extends andretracts from the interior rearview mirror assembly or from an accessorymodule or overhead console or the like within the vehicle cabin and inthe field of view of the driver of the vehicle. For example, the videodisplay may comprise a slide out video display such as described in PCTApplication No. PCT/US03/40611, filed Dec. 19, 2003 and published onJul. 15, 2004 as International Publication No. WO 2004/058540; and/orU.S. provisional application, Ser. No. 60/630,061, filed Nov. 22, 2004;Ser. No. 60/667,048, filed Mar. 31, 2005, which are hereby incorporatedherein by reference. The display screen thus may be automaticallyextended and activated in response to a detection of an object rearwardof the vehicle by the ultrasonic sensors, whereby the extension of thedisplay screen provides an alert to the driver so that the driver'sattention is readily drawn to the now extended and activated displayscreen. The imaging and display system and ultrasonic sensing system ofthe present invention thus may provide a display only when there is anobject detected and may provide an alert function to the driver to viewthe display when an object is detected. In such an application, thevideo display may not be extended and be activated in response to theshifting into reverse, but would extend and activate after detection ofan object exteriorly or rearward of the vehicle by the auxiliary sensoror sensors.

Although described above as ultrasonic sensors or sensing devices, it isenvisioned that the imaging and display system of the present inventionmay be combined with other types of sensing systems or auxiliary orsecondary or triggering or initial sensing devices or sensingtechniques. For example, the imaging and display system may be operablein conjunction with or in combination with radar devices, sonar devices,laser sensing or lidar devices or laser scanning devices or the like,without affecting the scope of the present invention. The initial ortriggering or auxiliary sensing device or devices may initiate ortrigger enhanced imaging or zooming or other characteristics of thecamera, or may initiate or trigger enhanced processing of the capturedimages, or may initiate or trigger enhanced display features or thelike, in response to a detection of an object rearward of the vehicle byone of the initial or triggering or auxiliary sensing devices. Althoughshown and described as being combined with or operating in conjunctionwith a rearward facing camera and associated display, it is envisionedthat aspects of the auxiliary sensing devices or system of the presentinvention may be equally suitable for use in conjunction with sidewardfacing imaging systems, such as side object detection systems or lanechange assist systems or the like, or forward facing imaging systems,such as lane departure warning systems, adaptive speed control, headlampcontrols, rain sensors or the like.

Optionally, and desirably, the image capture device or imaging devicemay be at least partially contained within an imaging module or cameramodule, which includes imaging sensor or imager and a lens positionedwithin a housing which defines a transparent window (which may comprisean at least substantially transparent glass or polycarbonate or acrylic(or other suitable material) window or panel) at the end of lens (suchas described in PCT Application No. PCT/US2003/036177, filed Nov. 14,2003 and published Jun. 3, 2004 as International Publication No. WO2004/047421, which is hereby incorporated herein by reference). Theimaging module may include the circuitry and controls or cameramicrocontroller and video encoder for the imaging sensor, such as on oneor more printed circuit boards contained within the housing.

Optionally, the camera module may comprise a fully sealed module, whichprotects the imaging device and microcontroller and circuitry fromexposure to the elements at the exterior of the vehicle, such as dust,dirt, mud, water, ice, salt and the like. The lens may provide part ofthe sealing of the module. For example, the camera module may include alens that may be purchased or obtained as a subassembly that is separatefrom the imaging sensor and mounted to the module. The lens may loadedinto the module, such as via insertion of the lens or threading of thelens into an aperture or opening in the module wall or adjacent to atransparent cap or cover of the module wall. The lens assembly (which istypically a five element lens set, but may include more or less opticelements without affecting the scope of the present invention) may bepositioned at the opening or aperture or cover such that the lens ispositioned at the appropriate or precise location relative to theimaging sensor or chip to properly focus the image onto the imagingsensor.

Optionally, the lens may be adjustably positioned, such as via threadinginto a threaded opening, to precisely position the lens to focus theimage onto the imaging sensor. Once positioned at the precise orappropriate location, the lens may be retained or secured in the preciseposition, and the module may then be sealed with a back plate and gasketto seal the lens and imaging sensor and associated microprocessor andcircuitry within the module. The module thus provides enhancedpositioning of the lens relative to the imaging sensor, since the lensmay be readily adjusted to provide proper focusing relative to a fixedimaging sensor.

The module may include a transparent cap or cover through which thefield of view of the imaging sensor and lens is directed. Thetransparent cap may comprise a molded polycarbonate material or the likeand may provide a substantially transparent and durable cover at thelens. Optionally, the transparent cap may include an anti-reflectivecoating or a hydrophobic or hydrophilic coating or the like, such as thecoatings described in PCT Application No. PCT/US2003/036177, filed Nov.14, 2003 and published Jun. 3, 2004 as International Publication No. WO2004/047421, which is hereby incorporated herein by reference. Thetransparent cap thus may provide functionality and sealing to the cameramodule.

Because the lens assembly may be selected and positioned at the imagingsensor as a separate sub-assembly or as an aftermarket sub-assembly, thelens assembly may be selected to provide the desired effect depending onthe particular application of the camera module. The lens may then beassembled to the module and the appropriate module, with the desiredlens and features or content, may be mounted to the vehicle andconnected to the existing connector (such as a five wire connector asdescribed above). The display system may further process the image dataor encoded signal to digitally correct for distortion in the image dueto lens distortion and the like.

The present invention thus provides a customized imaging system thatprovides various features or functions or modes or content to theimaging device or system without requiring costly communication or datawires connecting between the imaging device and the user inputs withinthe vehicle. The present invention thus provides a vision system that issuitable for economically configuring the system to the desired contentfor a particular application of the imaging system. A user thus mayselect a vision system with a particular camera and/or display, withlittle or no affect on the wiring or connections between the camera andthe display.

Optionally, the imaging system may be operable to function as a lanedeparture warning (LDW) system utilizing image processing of the imagescaptured by the rearward facing imaging sensor or camera. The imagingsystem may include controls and/or circuitry for operating as such alane departure warning system and thus may process the images to detectthe lane markers and the like along the road surface, or the imagingsystem may provide image data to a separate image processor ormicrocontroller for processing the image data to detect the lane markersand the like along the road surface. The imaging system may utilizeimage processing techniques such as those described in U.S. patentapplication Ser. No. 10/427,051, filed Apr. 30, 2003, now U.S. Pat. No.7,038,577; and/or Ser. No. 11/105,757, filed Apr. 14, 2005, now U.S.Pat. No. 7,526,103; and/or U.S. provisional application, Ser. No.60/607,963, filed Sep. 8, 2004; and/or Ser. No. 60/638,687, filed Dec.23, 2004, which are hereby incorporated herein by reference.

The imager or imaging sensor 22 of the imaging system 16 of the imagingsystem 10 may comprise an imaging array sensor or a pixelated imagingarray, such as a multi-pixel array such as a CMOS sensor or a CCD sensoror the like, such as the types disclosed in commonly assigned U.S. Pat.Nos. 5,550,677; 5,670,935; 5,796,094; 5,877,897; 6,097,023; 6,498,620;and 6,690,268, and U.S. patent application Ser. No. 09/441,341, filedNov. 16, 1999, now U.S. Pat. No. 7,339,149, which are herebyincorporated herein by reference, or such as an extended dynamic rangecamera, such as the types disclosed in PCT Application No.PCT/US2003/036177, filed Nov. 14, 2003 and published Jun. 3, 2004 asInternational Publication No. WO 2004/047421, which is herebyincorporated herein by reference. For example, the imaging sensor maycomprise a CMOS camera, such as the OV7930 single chip CMOS color NTSCcamera available from OmniVision Technologies Inc. of Sunnyvale, Calif.Such color cameras may have the performance characteristics identifiedabove and may additionally provide RGB and/or YCrCb video signals.Preferably, the color video camera operates at a minimum illumination(3000 K) of less than about 5 lux at f1.2, more preferably of less thanabout 3 lux at f1.2, and most preferably less than about of less thanabout 2 lux at f1.2. Such CMOS imaging sensors typically may have a peaksensitivity in the near infrared range, such as at approximately 850 nmto 900 nm.

Such pixelated imaging sensors may include a plurality of pixels, withat least some of the pixels masked or covered with a particular colorfilter, such that the individual pixels function to capture a particularcolor, such as red, green and blue colors or the like, such as disclosedin U.S. Pat. Nos. 5,550,677; 5,670,935; 5,796,094; 6,097,023; and/or6,498,620, referenced above. For example, the imaging sensor maycomprise an individual blue or a green or a red color filter over eachpixel element of the CMOS multi-pixel element array. The imaging sensoris thus operable to provide color images to the display. Such RGBfilters enable the capture of a color image by the CMOS detector, butnecessarily result in a reduced or decreased low light level sensitivityfor a color camera compared to a monochromatic or black and whitecamera. Optionally, and preferably, the imaging sensor may be capable ofselectively operating in either a color mode, in which a color image maybe displayed at display element 28 of display system 14, or amonochromatic or black and white mode, in which a monochromatic or blackand white image may be displayed at display element 28 of display system14, such as by utilizing aspects of the imaging sensor disclosed in U.S.Pat. No. 6,498,620; and/or PCT Application No. PCT/US2003/036177, filedNov. 14, 2003 and published Jun. 3, 2004 as International PublicationNo. WO 2004/047421, which are hereby incorporated herein by reference.

Although described as a CMOS type camera, clearly other types of imagingarrays or imaging sensors or cameras may be implemented with the imagingsystem of the present invention. For example, the imaging sensor maycomprise a CCD or other type of sensor, without affecting the scope ofthe present invention. Preferably, the selected imaging sensor has a lowdark current and thus provides enhanced ruggedness and enhancedperformance at higher temperatures. Optionally, the dark current (thecurrent through the pixels when they are not sensing light) may be usedto detect the temperature at the imaging sensor, such as described inU.S. patent application Ser. No. 11/105,757, filed Apr. 14, 2005, nowU.S. Pat. No. 7,526,103; and U.S. provisional application Ser. No.60/607,963, filed Sep. 8, 2004, which is hereby incorporated herein byreference.

Optionally, the imaging device or system may communicate the videosignals to the display device or system via other types of signalcommunicating means. For example, the imaging device or system maycommunicate the video signals to the display device or system via anLVDS output of the imaging device or system. Optionally, the imagingsystem and the display system may share common components or circuitryor a common microprocessor to reduce components and cost of the visionsystem.

Referring now to FIG. 6, an image capture system or imaging system 110is positioned at a vehicle, such as at or in an interior portion of thevehicle, such as at or in an accessory module or pod or attachment 112of the vehicle, and is operable to capture an image of a scene occurringexteriorly of the vehicle, such as forwardly of the vehicle. Imagingsystem 110 includes an image capture device or imaging device or imagingsystem or camera or sensor 114 that is directed exteriorly of thevehicle and has an exterior field of view 116. The images or framescaptured by image sensor 114 are processed by a control 118 to detectobjects or items of interest in the captured images. The control mayprocess the captured image data to determine if headlamps or taillightsof other vehicles are present in the images, such as for a headlampcontrol system, and/or the control may process the captured images todetect lane markers along the road surface, such as for a lane departurewarning system, and/or may process the image data to detect othercharacteristics or objects, as discussed below (such as by utilizingaspects of the imaging system described in U.S. provisional applicationSer. No. 60/618,686, filed Oct. 14, 2004, which is hereby incorporatedherein by reference in its entirety). The control may synchronize theprocessing techniques and the camera or image sensor settings to enhanceand/or optimize processing of images for the particular system orfunction, as also discussed below.

Imaging sensor 112 may be positioned at the vehicle and directed ororiented with a field of view generally exteriorly of the vehicle forcapturing images of the exterior scene for processing by the control, asdiscussed below. Imaging system 110 may utilize principles of othervehicle vision or imaging systems, such as a vision or imaging system orcontrol of the types disclosed in U.S. Pat. Nos. 6,757,109; 6,717,610;6,396,397; 6,201,642; 6,353,392; 6,313,454; 6,396,397; 5,550,677;5,670,935; 5,796,094; 5,877,897; 6,097,023; and 6,498,620, and U.S.patent application Ser. No. 09/441,341, filed Nov. 16, 1999, now U.S.Pat. No. 7,339,149, and Ser. No. 10/427,051, filed Apr. 30, 2003, nowU.S. Pat. No. 7,038,577, which are all hereby incorporated herein byreference. In a preferred embodiment, the imaging system 112 may includea lens element or optic between the imaging device 114 and the exteriorscene. The optic may comprise an asymmetric optic, which focuses agenerally central portion of the scene onto the imaging device, whileproviding classical distortion on the periphery of the scene or field ofview.

In the illustrated embodiment, the image sensor 112 is mounted at or inan accessory module or windshield electronics module or mirrorattachment or pod 120 and is arranged to have a field of view 116forward of the vehicle, such as through the windshield of the vehicleand preferably through a portion of the windshield that is wiped by awindshield wiper of the vehicle. The image sensor 112 thus may captureimages of a forward scene as the vehicle is traveling forwardly along aroad or street or highway or the like. Optionally, the imaging device orsensor may be positioned elsewhere, such as at or in the interiorrearview mirror assembly 122, or at or in an accessory module orwindshield electronics module or the like (as discussed below), withoutaffecting the scope of the present invention. Optionally, the imagesensor and imaging system may be operable to capture images of otherareas exteriorly of the vehicle to provide image data of other areasaround or near the vehicle, without affecting the scope of the presentinvention. The imaging system thus may be operable to capture images ofthe scene sidewardly of the vehicle for a blind spot or side objectdetection or lane change assist systems or immediately rearward of thevehicle to detect objects rearward of the vehicle, such as for assistingthe driver of the vehicle in backing up or maneuvering the vehicle inreverse.

As shown in FIG. 7, control 118 may control the imager or image sensor114 via a control signal 118 a, while the control may receive image datafrom the image sensor via an image data signal 114 a. The control ormicroprocessor 118 is operable to process the image data generated bythe image sensor 114 to analyze the image data and detect objects orlight sources of interest in the captured image. The control or imagingsystem may selectively process the captured image data to detect objectsor light sources or lane markers or the like in the exterior scene, andmay process particular portions or regions of the captured image inaccordance with the respective function of the control, as discussedbelow.

The control may be operable to control various accessories (or togenerate an output to a control or circuitry of one or more accessories)in response to the image processing. For example, the control maycontrol or adjust a headlamp high-low beam setting in response to adetection of headlamps or taillights in the captured images, and/or mayactuate an alert device, such as a visible display or audible alert orthe like, in response to a detection of the vehicle drifting out of itslane along the road, and/or may adjust an interior lighting or displayintensity of a light or display within the vehicle in response to anambient light detection, and/or may generate other outputs oractuate/control other accessories in response to detections of otherobjects or light sources of interest via such image processing.

The imaging system thus may be operable to function as or in conjunctionwith a lane departure warning system, and may generate an alert to thedriver of the vehicle in response to a detection that the vehicle isdrifting or moving out of a detected lane along the road. Such anapplication may utilize principles of systems of the types described inU.S. Pat. Nos. 6,353,392 and 6,313,454; and/or U.S. patent applicationSer. No. 10/427,051, filed Apr. 30, 2003, now U.S. Pat. No. 7,038,577;Ser. No. 11/105,757, filed Apr. 14, 2005, now U.S. Pat. No. 7,526,103;and/or Ser. No. 10/209,173, filed Jul. 31, 2002, now U.S. Pat. No.6,882,287, and/or U.S. provisional application, Ser. No. 60/607,963,filed Sep. 8, 2004; and/or Ser. No. 60/638,687, filed Dec. 23, 2004,which are hereby incorporated herein by reference. Optionally, theimaging system may capture and process images for a headlamp controlapplication, and may utilize principles of systems of the typesdescribed in U.S. Pat. Nos. 6,824,281; 5,796,094; 6,097,023; 5,320,176;and 6,559,435, which are hereby incorporated herein by reference, and/ormay capture and process images for a rain sensing function or rainsensor application, and may utilize principles of the systems of thetypes described in U.S. Pat. Nos. 6,824,281; 6,320,176; 6,353,392;6,313,454; 6,516,664; 6,341,523; and 6,250,148; and/or in U.S. patentapplication Ser. No. 10/348,514, filed Jan. 21, 2003, now U.S. Pat. No.6,968,736, which are all hereby incorporated herein by reference, and/ormay capture and process images for an ambient light detection or thelike, and may utilize principles of systems of the types described inU.S. Pat. No. 5,550,677 and/or 5,670,935, which are hereby incorporatedherein by reference, and/or may capture and process images for acollision avoidance system or vehicle separation system, and may utilizeprinciples of systems of the types described in U.S. Pat. Nos.6,411,204; 6,396,397; 6,124,647; 6,291,906; and 6,534,884, and/or U.S.patent application Ser. No. 10/422,512, filed Apr. 24, 2003, now U.S.Pat. No. 7,123,168, which are all hereby incorporated herein byreference, and/or may capture and process images for a navigationalsystem, and may utilize principles of systems of the types described inU.S. Pat. Nos. 6,477,464; 5,924,212; 4,862,594; 4,937,945; 5,131,154;5,255,442; 6,678,614; and/or 5,632,092, and/or U.S. patent applicationSer. No. 10/456,599, filed Jun. 6, 2003, now U.S. Pat. No. 7,004,593;Ser. No. 10/645,762, filed Aug. 20, 2003, now U.S. Pat. No. 7,167,796;Ser. No. 11/105,757, filed Apr. 14, 2005, now U.S. Pat. No. 7,526,103;and Ser. No. 10/422,378, filed Apr. 24, 2003, now U.S. Pat. No.6,946,978, now U.S. Pat. No. 6,946,978; and/or PCT Application No.PCT/US03/40611, filed Dec. 19, 2003 and published Jul. 15, 2004 asInternational Publication No. WO 2004/058540; and/or PCT Application No.PCT/US04/015424, filed May 18, 2004 and published Dec. 2, 2004 asInternational Publication No. WO 2004/103772, and/or U.S. provisionalapplication Ser. No. 60/607,963, filed Sep. 8, 2004, which are allhereby incorporated herein by reference. Other accessories or functionsor applications may also or otherwise be provided or controlled oradjusted by the imaging system, without affecting the scope of thepresent invention.

The control may adjust one or more characteristics or settings of theimage sensor or camera and/or may operate an iris and/or an optical zoomand/or a digital zoom or the like so that the image sensor is adapted orset to enhance image capturing for a particular function of the imagingsystem or control. Optionally, the control may adjust a focus of theimage sensor (such as via adjustment of the lens or optic) to provide aclear captured image focused on the desired or appropriate objects forthe particular application of the imaging system.

The imager or imaging sensor 114 of the imaging system 110 may comprisean imaging array sensor or a pixelated imaging array, such as amulti-pixel array such as a CMOS sensor or a CCD sensor or the like,such as the types disclosed in commonly assigned U.S. Pat. Nos.5,550,677; 5,670,935; 5,796,094; 5,877,897; 6,097,023; 6,498,620; and6,690,268, and U.S. patent application Ser. No. 09/441,341, filed Nov.16, 1999, now U.S. Pat. No. 7,339,149, which are hereby incorporatedherein by reference. For example, the multi-pixel array may comprise amulti-pixel array having a 640×480 pixel array or the like. In order toreduce the computational load on the processor, only a desired subset orzone of the multi-pixel array frame may be chosen to be processed, suchas for a particular application of the imaging system. For differentapplications, the objects or light sources of interest may be differentobjects or categories or classifications of objects and/or may betypically located at different regions of the captured image. Forexample, for a headlamp control application or function, the objects ofinterest are light sources, particularly headlamps and taillights, thatare typically located at a generally central band or region of interestacross the captured images, while for a lane departure warningapplication or function, the objects of interest are lane markers orroad edges or the like that are typically located along a lower regionof interest of the captured images (such as along the lower about 30percent of the captured image or thereabouts). For ambient lightdetection, the light or region of interest would typically be along theupper region of the captured images (such as about the upper 20 percentof the captured image or thereabouts). The desired or appropriate orchosen subset or zone or target area of the multi-pixel array frame maybe processed for the particular application to reduce processing orcomputational load when additional processing of other subsets or zonesmay not enhance the performance of the imaging system for thatparticular application.

Also, for different applications, it may be desirable to capture therespective images with the image sensor being at a different sensitivityor setting. For example, a camera setting or characteristic, such as anexposure time, an amplification, a gain, a white balance and/or thelike, of the image sensor may be adjusted for capturing images tooptimize the captured image and image data for a particular applicationor function. Amongst other things, the sensitivity of the image sensorcan be adjusted by adjusting the integration time of the pixels and theamplification stage in reading out pixels. This allows the image sensorto be adapted to various lighting conditions at the observed exteriorscene. In a closed loop camera control scheme, the image sensorsensitivity can be adjusted as determined by an application module sothat the image brightness meets the application specific requirements.For example, the control may adjust the image sensor to be at anincreased sensitivity for detecting headlamps and taillights duringnighttime conditions, yet may decrease the sensitivity for detectinglane markers along the road surface, particularly during nighttimeconditions when the headlamps are activated and the road is thus wellilluminated. Other adjustments or settings or characteristics, such as adesired or selected zone of interest in a frame or an optical zoom ordigital zoom or panning of or through a frame or the like, may be madeor set or adjusted to optimize the image sensor for capturing one ormore images for a particular function or accessory control.

The control thus may adjust a characteristic or setting of the imagingsensor for particular captured images and may process the image data ofthose particular captured images in a manner or processing techniquesuited for a particular application or function of the control, such asfor headlamp control, lane departure warning, rain sensing, vehicledetection, vehicle lighting and/or display control, and/or the like. Thecontrol may synchronize the camera settings for particular capturedimages with the processing technique applied to the image dataassociated with those captured images, and may switch or alternatebetween different settings and processing techniques to provide thedesired processing and analysis and output for the desired applications.

For example, and with reference to FIGS. 8 and 9, the control mayselectively or occasionally or intermittently process images for firstand second applications, such as a lane departure warning system and aheadlamp control or the like. The control thus may adjust a camerasetting or settings or sensitivity or other characteristic or parameterof the image sensor for capturing a first set of images and may processthe image data from the first set of captured images in a first manner,and may generate a first output in response to the first imageprocessing. The control may also adjust a camera setting or settings orsensitivity or other characteristic or parameter of the image sensor forcapturing a second set of images and may process the image data from thesecond set of captured images in a second manner, and may generate asecond output in response to the first image processing. The first andsecond camera settings/characteristics and/or the first and secondprocessing manners or techniques may be different from one another tooptimize the capturing and processing of the image data for theparticular first or second application or function. The control mayfurther adjust the camera settings and image processing techniques for athird application or more applications as desired, without affecting thescope of the present invention. The first and second (and third and moreif applicable) settings and processing techniques are synchronized suchthat each image captured by the image sensor at one of the settings isprocessed by the control or processor at the appropriate orcorresponding processing technique for the associated application orfunction or accessory control.

Multiple applications thus can be operated and controlled using the sameimager and processing resources of the imaging system. In a case whereonly two applications share the system resources and where frames ofimage data are thus processed alternately by the two application modulesusing the same processor, the control may alternate every other imagefor the two applications. For example (and as shown in FIG. 9), images1, 3, 5, etc. may be captured with the image sensor settings at aparticular setting and may be processed according to a particularprocessing technique, while images 2, 4, 6, etc. may be captured withthe image sensor settings at a different setting and/or may be processedaccording to a different processing technique. Clearly, however, otheriterations and alternate capturing and/or processing of images may beimplemented without affecting the scope of the present invention.Although shown as separate applications in FIG. 9, the same imageprocessor may process the different sets of images for the respectivefirst and second applications of the imaging system.

Optionally, such as for cases where the two applications do not have toprocess images at the same frame rate, the frame rate ratio can beadjusted accordingly. For example, if the imager is operable to provideup to 15 frames per second (fps), and the first application needs thecontrol to process five frames per second while the second or otherapplication only needs the control to process one frame per second, thenthe control may process five frames in a row for the first application,and may then process only one frame for the second application. Thisprocessing scheme may be adjusted for any desired frames per second andmay be extended to allow more than two applications to execute on thesystem, without affecting the scope of the present invention.

As shown in FIG. 8, the control or image processor 118 may selectivelyprocess the image data to detect a desired object or light source ofinterest. For example, the control 118 may process image data to detectheadlamps (as shown at 126 in FIG. 8), and may generate an output to aheadlamp control 128 to adjust or control the headlamps of the vehicle(or the control 118 may adjust or control the headlamps of the vehiclein response to the image processing). Optionally, the control 118 mayalso or otherwise selectively process the image data to detect lanemarkers (as shown at 130 in FIG. 8), and may generate an output to alane departure warning system or to an alert device 132 to alert thedriver of a lane drift or the like as the vehicle travels along theroad. Optionally, the control 118 may also or otherwise selectivelyprocess the image data to determine an ambient light level (as shown at134 in FIG. 8), and may generate an output to a display 136 or to aninterior light control 138 or the like to adjust an intensity of thedisplay or lights in response to such image processing. Other capturedimages and/or other objects or light sources or light levels of interestmay also or otherwise be analyzed to provide other outputs to or controlof other functions or accessories of the vehicle, without affecting thescope of the present invention. Each of the image processing techniquesapplied to the image data may be synchronized with the settings orsensitivity of the image sensor to optimize the performance anddetection capabilities of the imaging system for the particularapplications associated with or conducted by the imaging system.

The present invention thus provides an imaging system that mayselectively capture images via an image sensor and may selectivelyprocess the captured images to determine or detect a desired orappropriate characteristic of the captured images. The imaging systemmay synchronize a setting or sensitivity or focus of the image sensorwith an image processing technique so that the captured image data areselectively captured and/or processed in a particular manner for aparticular application of the imaging system. The imaging system thusmay provide multiple functions or applications by utilizing a commonimaging sensor and common image processor, and thus may provide amulti-tasking imaging system at a reduced cost and with reducedcomponents. The imaging system may capture multiple frames sequentially,and may elect (such as in a predetermined manner) the processing ortreatment of the respective frames or sets of frames. The imaging systemthus may focus or harness the processing power to a restricted set ofcriteria for one subset of frames, and to another restricted set ofcriteria for another subset of frames. The imaging system of the presentinvention thus may selectively focus or harness the image sensorcapturing capabilities and the image processing power to enhance theimage capturing and processing of different frames or sets of frames fordifferent applications.

Although shown and described as a forward facing imaging sensor, it isenvisioned that aspects of the present invention may be suitable forselectively capturing and/or processing image data with a rearwardfacing imaging sensor and control or image processor. For example, therearward facing imaging system may be operable to function as a lanedeparture warning (LDW) system utilizing image processing of the imagescaptured by a rearward facing imaging sensor or camera. The imagingsystem may include controls and/or circuitry for operating as such alane departure warning system and thus may process the images to detectthe lane markers and the like along the road surface, or the imagingsystem may provide image data to a separate image processor ormicrocontroller for processing the image data to detect the lane markersand the like along the road surface. The imaging system may utilizeimage processing techniques such as those described in U.S. patentapplication Ser. No. 10/427,051, filed Apr. 30, 2003, now U.S. Pat. No.7,038,577; and/or Ser. No. 11/105,757, filed Apr. 14, 2005, now U.S.Pat. No. 7,526,103; and/or U.S. provisional application, Ser. No.60/607,963, filed Sep. 8, 2004; and/or Ser. No. 60/638,687, filed Dec.23, 2004, which are hereby incorporated herein by reference. Optionally,the imaging system may selectively capture and process image data of therearward facing image sensor to provide a backup aid function, such asby utilizing principles described in U.S. Pat. Nos. 5,550,677;5,760,962; 5,670,935; 6,201,642; 6,717,610 and/or 6,757,109, and/or inU.S. patent application Ser. No. 10/418,486, filed Apr. 18, 2003, nowU.S. Pat. No. 7,005,974, and/or in PCT Application No.PCT/US2003/036177, filed Nov. 14, 2003 and published Jun. 3, 2004 asInternational Publication No. WO 2004/047421, which are all herebyincorporated herein by reference.

The imager or imaging sensor 114 of the imaging system 110 may comprisean imaging array sensor or a pixelated imaging array, such as amulti-pixel array such as a CMOS sensor or a CCD sensor or the like,such as the types disclosed in commonly assigned U.S. Pat. Nos.5,550,677; 5,670,935; 5,796,094; 5,877,897; 6,097,023; 6,498,620; and6,690,268, and U.S. patent application Ser. No. 09/441,341, filed Nov.16, 1999, now U.S. Pat. No. 7,339,149, which are hereby incorporatedherein by reference, or such as an extended dynamic range camera, suchas the types disclosed in PCT Application No. PCT/US2003/036177, filedNov. 14, 2003 and published Jun. 3, 2004 as International PublicationNo. WO 2004/047421, which is hereby incorporated herein by reference.For example, the imaging sensor may comprise a CMOS camera, such as theOV7930 single chip CMOS color NTSC camera available from OmniVisionTechnologies Inc. of Sunnyvale, Calif. Such color cameras may have theperformance characteristics identified above and may additionallyprovide RGB and/or YCrCb video signals. Preferably, the color videocamera operates at a minimum illumination (3000 K) of less than about 5lux at f1.2, more preferably of less than about 3 lux at f1.2, and mostpreferably less than about of less than about 2 lux at f1.2. Such CMOSimaging sensors typically may have a peak sensitivity in the nearinfrared range, such as at approximately 850 nm to 900 nm orthereabouts.

Such pixelated imaging sensors may include a plurality of pixels, withat least some of the pixels masked or covered with a particular colorfilter, such that the individual pixels function to capture a particularcolor, such as red, green and blue colors or the like, such as disclosedin U.S. Pat. Nos. 5,550,677; 5,670,935; 5,796,094; 6,097,023; and6,498,620, referenced above. For example, the imaging sensor maycomprise an individual blue or a green or a red color filter over eachpixel element of the CMOS multi-pixel element array. The imaging sensoris thus operable to detect colors, which is useful in determiningwhether a detected object or item is a headlamp or a taillight, such asdescribed in U.S. Pat. No. 5,796,094, referenced above.

Although described as a CMOS type camera, clearly other types of imagingarrays or imaging sensors or cameras may be implemented with the imagingsystem of the present invention. For example, the imaging sensor maycomprise a CCD or other type of sensor, without affecting the scope ofthe present invention. Preferably, the selected imaging sensor has a lowdark current and thus provides enhanced ruggedness and enhancedperformance at higher temperatures. Optionally, the dark current (thecurrent through the pixels when they are not sensing light) may be usedto detect the temperature at the imaging sensor, such as described inU.S. patent application Ser. No. 11/105,757, filed Apr. 14, 2005, nowU.S. Pat. No. 7,526,103; and/or U.S. provisional application Ser. No.60/607,963, filed Sep. 8, 2004, which is hereby incorporated herein byreference.

Optionally, a method or system may be implemented to transform a CMOScolor imager from its individual color space into a standardized colorprotocol, such as the CIE color space, in order to enhance theidentification of a detected object or light source. The CentreInternationale dEclairage (CIE) is an international organization thatestablishes methods for measuring color and that recommends standardsand procedures for light and lighting, including colorimetry. The colorstandards for colormetric measurements are internationally acceptedspecifications that define color values mathematically. CIE definescolor as a combination of three axes: x, y, and z. CIE color models areconsidered device independent because the colors should not differ,theoretically, from one output device to another if properly calibrated.

The CIE color spaces thus provide the foundation upon whichdevice-independent color and color management are built. Deviceprofiles, device calibration, and gamut mapping may also be needed forcolor management. A device profile describes a device's colorcapabilities including color gamut, color production method, and deviceoperation modes. Device profiles are typically created by color imagingscientists using spectrophotometers, which are instruments that measurethe relative intensities of light in different parts of the visiblespectrum. The measurements are then used to produce the device profile,such as via various algorithms and the like. Device profiles are used bythe color management software to translate color data from one device toanother based upon an independent color space or system for orderingcolors that respects the relationships or similarity among them.

The CMOS color imager of the imaging system of the present invention isused in an automotive environment where all the road scene objects ofinterest that may be typically encountered may be classified within theCIE Color System. Digital imaging systems represent color in any givennumber of ways. The transformation provides an easy method for theidentification of CIE color. Classification of the various objectswithin the road scene can be enhanced using CIE color as well as otherimage processing methods to provide high confidence that any particulardetected object or light source is correctly identified.

The transformation requires an initial characterization of the pixeloutput of the CMOS imager using four primary calibration targets and aspectroradiometer. The CMOS imager is used to measure the “color” of thecalibrated targets. The calibrated targets are also measured using thespectroradiometer. The transformation yields a three by three matrixthat is applied to the x, y, z values of the camera's output to provideCIE color. The CIE color of the detected object or light source may thenbe compared to the classifications or index of the various objects orlight sources of interest to identify the detected object or lightsource.

Therefore, the present invention provides an imaging system thatincludes an imaging device and a control that is operable to selectivelycapture images and process image data to provide multiple functions orapplications of a common image sensor and image processor. The imagingsystem may synchronize the settings of the image sensor with theprocessing techniques applied to the captured image data to optimize twoor more functions of the imaging system. The present invention thusprovides a multi-tasking capability to a common image sensor and imageprocessor while providing enhanced capturing of the images andprocessing of the image data for the desired functions or applications.

Optionally, an imaging sensor or system of the present invention may beassociated with an imaging and display system for displaying thecaptured images to the driver of the vehicle. In such displayapplications, a variety of display means or display devices may beutilized to visually convey information to the driver of the vehicle,without affecting the scope of the present invention. For example, andsuch as described in U.S. Pat. No. 6,477,464, which is herebyincorporated herein by reference, a text display may be provided and/oran iconistic display may be provided, such as a display readable throughthe interior rearview mirror reflective element itself. In this regard,use of a transflective or display on demand (DOD) type display (such asdisclosed in commonly assigned, U.S. patent application Ser. No.10/054,633, filed Jan. 22, 2002, now U.S. Pat. No. 7,195,381, and inU.S. Pat. Nos. 6,690,268; 5,668,663 and 5,724,187, the entiredisclosures of which are hereby incorporated by reference herein), maybe preferred. For example, a video display element or a video displayscreen or an information display element can be used (such as anelongated alphanumeric/multi-pixel/multi-icon display element and/orsuch as an LCD display or an emitting display element, such as amulti-pixel electroluminescent display or field emission display orlight emitting diode display (organic or inorganic) or the like) whichis disposed within the mirror housing of the interior mirror assembly ofthe vehicle, and located behind the mirror reflective element in themirror housing, and configured so that the information displayed by thedisplay element (that is positioned to the rear of the reflector of themirror reflective element) is viewable by the driver through the mirrorreflective element. Such a display can be accomplished by partially orwholly removing the reflector in the area of the display or, morepreferably, by providing a display on demand type display, whereby thereflective element comprises a transflective element, as discussedbelow.

Note that other display locations are possible for display of the videoimage or information display, such as a text message or the like, to thedriver or occupant of the vehicle. For example, a video image may bedisplayed on an LCD video screen of flip-down display (such as isdisclosed in U.S. Pat. No. 6,690,268, incorporated above), or on a videoscreen incorporated into the rearview mirror assembly (such as the typesdescribed in U.S. patent application Ser. No. 10/964,512, filed October13, now U.S. Pat. No. 7,308,341, and/or in PCT Application No.PCT/US03/40611, filed Dec. 19, 2003 and published Jul. 15, 2004 asInternational Publication No. WO 2004/058540, which is herebyincorporated herein by reference). Optionally, for example, a videodisplay located in the front instrument panel can be used, or a videodisplay located in an overhead console (such as an overhead accessorymodule or system as described in PCT Application No. PCT/US03/40611, andpublished Jul. 15, 2004 as International Publication No. WO 2004/058540,which is hereby incorporated herein by reference) can be used, withoutaffecting the scope of the present invention. Alternately, a low cost,multi-pixel display (such as the type disclosed in U.S. patentapplication Ser. No. 10/418,486, filed Apr. 18, 2003, now U.S. Pat. No.7,005,974, which is hereby incorporated herein by reference), such as alow cost multi-pixel vacuum fluorescent display, a low cost multi-pixelorganic light emitting diode (OLED), a low cost multi-pixel fieldemission display, or any other or similar multi-pixel light emittingdisplay or the like may be utilized, without affecting the scope of thepresent invention.

Optionally, the imaging sensor and control, and optionally a displaydevice and/or an audible alert device or speaker or the like, mayinclude a display element positioned at a rear portion of the vehicle,such as at the rear window of the vehicle or the like, on which thecaptured image is projected, such as via a projector within the vehicle.The displayed or projected image may be viewed at the rear portion ofthe vehicle by the driver of the vehicle, such as by viewing the imagein the rearview mirror or by turning to view the rearward image. Theimage thus may be projected as a reverse image or mirror image displaydepending on the particular application or desired viewing of thedisplayed/projected image.

Optionally, the display device and/or an audible alert device or speakermay be positioned at or in or near the interior rearview mirror assemblyof the vehicle. The mirror assembly may comprise a prismatic mirrorassembly, such as a prismatic mirror assembly utilizing aspectsdescribed in U.S. Pat. Nos. 6,318,870; 5,327,288; 4,948,242; 4,826,289;4,436,371; and 4,435,042; and PCT Application No. PCT/US04/015424, filedMay 18, 2004 and published Dec. 2, 2004 as International Publication No.WO 2004/103772; and U.S. patent application Ser. No. 10/933,842, filedSep. 3, 2004, now U.S. Pat. No. 7,249,860, which are hereby incorporatedherein by reference. Optionally, the prismatic reflective element maycomprise a conventional prismatic reflective element or prism or maycomprise a prismatic reflective element of the types described in PCTApplication No. PCT/US03/29776, filed Sep. 19, 2003 and published Apr.1, 2004 as International Publication No. WO 2004/026633; U.S. patentapplication Ser. No. 10/709,434, filed May 5, 2004, now U.S. Pat. No.7,420,756; and Ser. No. 10/993,302, filed Nov. 19, 2004, now U.S. Pat.No. 7,338,177; and U.S. provisional application Ser. No. 60/525,952,filed Nov. 26, 2003, which are all hereby incorporated herein byreference, without affecting the scope of the present invention. Avariety of mirror accessories and constructions are known in the art,such as those disclosed in U.S. Pat. Nos. 5,555,136; 5,582,383;5,680,263; 6,227,675; 6,229,319; and 6,315,421 (the entire disclosuresof which are hereby incorporated by reference herein), that can benefitfrom or be implemented with the present invention.

Optionally, the interior rearview mirror assembly may comprise anelectro-optic or electrochromic reflective element or cell, such as anelectrochromic mirror assembly and electrochromic reflective elementutilizing principles disclosed in commonly assigned U.S. Pat. Nos.6,690,268; 5,140,455; 5,151,816; 6,178,034; 6,154,306; 6,002,544;5,567,360; 5,525,264; 5,610,756; 5,406,414; 5,253,109; 5,076,673;5,073,012; 5,117,346; 5,724,187; 5,668,663; 5,910,854; 5,142,407; and/or4,712,879, which are hereby incorporated herein by reference, and/or asdisclosed in the following publications: N. R. Lynam, “ElectrochromicAutomotive Day/Night Mirrors”, SAE Technical Paper Series 870636 (1987);N. R. Lynam, “Smart Windows for Automobiles”, SAE Technical Paper Series900419 (1990); N. R. Lynam and A. Agrawal, “Automotive Applications ofChromogenic Materials”, Large Area Chromogenics: Materials and Devicesfor Transmittance Control, C. M. Lampert and C. G. Granquist, EDS.,Optical Engineering Press, Wash. (1990), which are hereby incorporatedby reference herein. The mirror assembly may include one or moredisplays, such as the types disclosed in U.S. Pat. No. 5,530,240 and/or6,329,925, which are hereby incorporated herein by reference, and/ordisplay-on-demand or transflective type displays, such as the typesdisclosed in U.S. Pat. Nos. 6,690,268; 5,668,663 and/or 5,724,187,and/or in U.S. patent application Ser. No. 11/021,065, filed Dec. 23,2004, now U.S. Pat. No. 7,255,451; Ser. No. 10/528,269, filed Mar. 17,2005, now U.S. Pat. No. 7,274,501; and/or Ser. No. 10/993,302, filedNov. 19, 2004, now U.S. Pat. No. 7,338,177; Ser. No. 10/533,762, filedMay 4, 2005, now U.S. Pat. No. 7,184,190; Ser. No. 11/226,628, filedSep. 14, 2005 and published Mar. 23, 2006 as U.S. Publication No.US-2006-0061008; and/or Ser. No. 10/054,633, filed Jan. 22, 2002, nowU.S. Pat. No. 7,195,381; PCT Application No. PCT/US03/29776, filed Sep.9, 2003 and published Apr. 1, 2004 as International Publication No. WO2004/026633 A2; and/or PCT Application No. PCT/US03/40611, filed Dec.19, 2003 and published Jul. 15, 2004 as International Publication No. WO2004/058540 A3, and/or U.S. provisional application, Ser. No.60/681,250, filed May 16, 2005; Ser. No. 60/690,400, filed Jun. 14,2005; Ser. No. 60/695,149, filed Jun. 29, 2005, which are all herebyincorporated herein by reference.

Optionally, the image sensor and control, and optionally a displaydevice and/or alert device, may be positioned at or in an accessorymodule or windshield electronic module of the vehicle, without affectingthe scope of the present invention. The accessory module may compriseany type of accessory module or windshield electronics module orconsole, such as the types described in U.S. patent application Ser. No.10/355,454, filed Jan. 31, 2003, now U.S. Pat. No. 6,824,281; and Ser.No. 10/456,599, filed Jun. 6, 2003, now U.S. Pat. Nos. 7,004,593, and/or6,690,268; 6,250,148; 6,341,523; 6,593,565; and 6,326,613, and/or PCTApplication No. PCT/US03/40611, filed Dec. 19, 2003 and published Jul.15, 2004 as International Publication No. WO 2004/058540 A3, which areall hereby incorporated herein by reference.

Optionally, the imaging system may be selectively operable andswitchable to provide the desired function and/or display to the driverof the vehicle. Such switching may occur in response to one or more userinputs in the vehicle, such as at the interior rearview mirror system ofthe vehicle or the like. The user inputs may be positioned at or near adisplay, such as along a perimeter region of a flip out or slide outvideo display of a mirror assembly or along a chin or bezel region ofthe mirror assembly, and may comprise buttons, switches, proximitysensors, touch sensors, or the like, without affecting the scope of thepresent invention. For example, the user inputs may comprise buttons orswitches, such as user inputs of the types described in PCT ApplicationNo. PCT/US2003/036177, filed Nov. 14, 2003 and published Jun. 3, 2004 asInternational Publication No. WO 2004/047421, and/or PCT Application No.PCT/US04/015424, filed May 18, 2004 and published Dec. 2, 2004 asInternational Publication No. WO 2004/103772, which are herebyincorporated herein by reference, touch sensors or proximity sensinginputs or the like, such as sensors of the types described in U.S. Pat.Nos. 6,001,486; 6,310,611; 6,320,282; 6,627,918; and 5,594,222; and/orU.S. Pat. Publication No. 2002/0044065, published Apr. 18, 2002 by Quistet al., now U.S. Pat. No. 7,224,324; and/or U.S. patent application Ser.No. 10/933,842, filed Sep. 3, 2004, now U.S. Pat. No. 7,249,860; and/orPCT Application No. PCT/US03/40611, filed Dec. 19, 2003 and publishedJul. 15, 2004 as International Publication No. WO 2004/058540 A3, whichare hereby incorporated herein by reference, or may comprise inputsmolded within the bezel of the mirror assembly, such as described inU.S. patent application Ser. No. 11/029,695, filed Jan. 5, 2005, nowU.S. Pat. No. 7,253,723, and/or U.S. provisional application, Ser. No.60/535,559, filed Jan. 9, 2004; Ser. No. 60/553,517, filed Mar. 16,2004; Ser. No. 60/556,259, filed Mar. 25, 2004, which are herebyincorporated herein by reference, and/or other types of buttons orinputs at a bezel region, such as the types described in U.S.provisional application, Ser. No. 60/690,401, filed Jun. 14, 2005,and/or Ser. No. 60/719,482, filed Sep. 22, 2005, which are herebyincorporated herein by reference, or may comprise membrane typeswitches, such as described in U.S. patent application Ser. No.11/140,396, filed May 27, 2005, now U.S. Pat. No. 7,360,932, and/or U.S.provisional application Ser. No. 60/575,904, filed Jun. 1, 2004; andSer. No. 60/624,320, filed Nov. 2, 2004, which are hereby incorporatedherein by reference; and/or the like, without affecting the scope of thepresent invention. The switching or inputs may control or adjust thecontrol and/or imaging sensor, such as by utilizing the principlesdescribed above and in U.S. provisional application Ser. No. 60/614,644,filed Sep. 30, 2004, which is hereby incorporated herein by reference.

Therefore, the present invention provides an imaging and display systemthat includes an imaging device and a display device connected via videoconnections or communication links. The imaging device includes userinputs and the imaging and display system is operable to selectivelycontrol or adjust the imaging device in response to the user inputs. Theimaging and display system is operable to communicate imaging signalsfrom the imaging device to the display device and to communicate cameracontrol signals from the display device to the imaging device alongcommon connections or wiring or communication links between the imagingdevice and the display device. The imaging and display system thus mayprovide various optional features or functions or modes without havingto change or add wiring or connections between the imaging device andthe display device.

Changes and modifications to the specifically described embodiments maybe carried out without departing from the principles of the presentinvention, which is intended to be limited only by the scope of theappended claims as interpreted according to the principles of patentlaw.

What is claimed is:
 1. A driver assistance system for a vehicle, saiddriver assistance system comprising: an imager disposed at a vehicleequipped with said driver assistance system and viewing one of (i)forward of the equipped vehicle and (ii) rearward of the equippedvehicle; said imager having a field of view exterior the equippedvehicle and operable to capture image data; said imager comprising (i) alens, (ii) a spectral filter and (iii) an image sensor comprising a CMOSphotosensor array of photosensor elements; a control comprising an imageprocessor; wherein said control is operable to process captured imagedata for at least a first application and a second application, andwherein said first application comprises lane departure warning and saidsecond application comprises headlamp control; wherein said control isoperable to adjust said image sensor to at least two settings includingat least a first setting and a second setting; wherein said control isoperable to process captured image data via at least two processingtechniques; wherein said control is operable to synchronize said atleast two settings of said image sensor and said at least two processingtechniques to extract information from said captured image data for saidfirst and second applications; wherein said control adjusts a setting ofsaid image sensor to said first setting to capture image data suitablefor said first application; wherein said control adjusts a setting ofsaid image sensor to said second setting to capture image data suitablefor said second application; wherein said image processor processescaptured image data captured by said image sensor at said first settingvia a processing technique suitable for said first application; whereinsaid image processor processes captured image data captured by saidimage sensor at said second setting via a processing technique suitablefor said second application; wherein said driver assistance systemcomprises an auxiliary sensor; wherein said auxiliary sensor is selectedfrom the group consisting of (i) an ultrasonic sensor, (ii) a radarsensor and (iii) a lidar sensor; wherein said auxiliary sensor isoperable to detect an object exterior the equipped vehicle; and wherein,responsive at least in part to detection of the object by said auxiliarysensor, at least one of (i) a setting of said image sensor is adjustedand (ii) processing of image data that corresponds to a region at whichthe object is detected is enhanced.
 2. The driver assistance system ofclaim 1, and wherein said first application also includes an applicationselected from the group consisting of (i) vehicle detection and (ii)object detection.
 3. The driver assistance system of claim 2, whereinsaid second application also includes an application selected from thegroup consisting of (i) driving separation indication, (ii) backupassist and (iii) object detection.
 4. The driver assistance system ofclaim 1, wherein said control is operable to process captured image datafor said first application from a first set of captured image data in afirst image processing to generate a first output in response to thefirst image processing, and wherein said control is operable to processcaptured image data for said second application from a second set ofcaptured image data in a second image processing to generate a secondoutput in response to the second image processing.
 5. The driverassistance system of claim 1, wherein, responsive to processing by saidcontrol of captured image data, said control determines an ambient lightlevel exterior the equipped vehicle.
 6. The driver assistance system ofclaim 1, wherein said control is operable to process captured image datafor a third application.
 7. The driver assistance system of claim 1,wherein said imager and at least a part of said control are disposed inan accessory module of the equipped vehicle and wherein said imagerviews forward of the equipped vehicle through a windshield of theequipped vehicle.
 8. The driver assistance system of claim 1, whereinimage data captured by said imager are output via LVDS.
 9. The driverassistance system of claim 1, wherein the frame rate used to captureimage data for said first application differs from the frame rate usedto capture image data for said second application.
 10. The driverassistance system of claim 1, wherein said control is operable to atleast one of (a) provide enhanced imaging of an object viewed by saidimager and (b) provide enhanced processing of at least a portion of animage captured by said imager that corresponds to a region exterior theequipped vehicle at which an object is viewed by said imager.
 11. Thedriver assistance system of claim 1, wherein said control is operable toselectively or intermittently or occasionally adjust a setting of saidimage sensor to extract information from said captured image data forheadlamp control and for lane departure warning.
 12. The driverassistance system of claim 11, wherein said control is operable toselectively or intermittently or occasionally adjust a processingtechnique to extract information from said captured image data forheadlamp control and for lane departure warning.
 13. A driver assistancesystem for a vehicle, said driver assistance system comprising: animager disposed at a vehicle equipped with said driver assistance systemand viewing one of (i) forward of the equipped vehicle through awindshield of the equipped vehicle and (ii) rearward of the equippedvehicle; said imager having a field of view exterior the equippedvehicle and operable to capture image data; said imager comprising (i) alens, (ii) a spectral filter and (iii) an image sensor comprising a CMOSphotosensor array of photosensor elements; a control comprising an imageprocessor; wherein said control is operable to process captured imagedata for at least a first application and a second application, andwherein said first application comprises lane departure warning and saidsecond application comprises headlamp control; wherein said control isoperable to adjust said image sensor to at least two settings includingat least a first setting and a second setting; wherein said control isoperable to process captured image data via at least two processingtechniques; wherein said control is operable to synchronize said atleast two settings of said image sensor and said at least two processingtechniques to extract information from said captured image data for saidfirst and second applications; wherein said control adjusts a setting ofsaid image sensor to said first setting to capture image data suitablefor said first application; wherein said control adjusts a setting ofsaid image sensor to said second setting to capture image data suitablefor said second application; wherein said image processor processescaptured image data captured by said image sensor at said first settingvia a processing technique suitable for said first application; whereinsaid image processor processes captured image data captured by saidimage sensor at said second setting via a processing technique suitablefor said second application; wherein said control is operable to processcaptured image data for a third application; wherein said driverassistance system comprises an auxiliary sensor; wherein said auxiliarysensor is selected from the group consisting of (i) an ultrasonicsensor, (ii) a radar sensor and (iii) a lidar sensor; wherein saidauxiliary sensor is operable to detect an object exterior the equippedvehicle; and wherein, responsive at least in part to detection of theobject by said auxiliary sensor, at least one of (i) a setting of saidimage sensor is adjusted and (ii) processing of image data thatcorresponds to a region at which the object is detected is enhanced. 14.The driver assistance system of claim 13, wherein said control isoperable to at least one of (a) provide enhanced imaging of an objectviewed by said imager and (b) provide enhanced processing of at least aportion of an image captured by said imager that corresponds to a regionexterior the equipped vehicle at which an object is viewed by saidimager.
 15. The driver assistance system of claim 13, wherein at leastone of (a) said control is operable to selectively or intermittently oroccasionally adjust a setting of said image sensor to extractinformation from said captured image data for headlamp control and forlane departure warning and (b) wherein said control is operable toselectively or intermittently or occasionally adjust a processingtechnique to extract information from said captured image data forheadlamp control and for lane departure warning.
 16. A driver assistancesystem for a vehicle, said driver assistance system comprising: animager disposed at a vehicle equipped with said driver assistance systemand viewing forward of the equipped vehicle through a windshield of theequipped vehicle; said imager having a field of view exterior theequipped vehicle and operable to capture image data; said imagercomprising (i) a lens, (ii) a spectral filter and (iii) an image sensorcomprising a CMOS photosensor array of photosensor elements; a controlcomprising an image processor; wherein said control is operable toprocess captured image data for at least a first application and asecond application, and wherein said first application comprises lanedeparture warning and said second application comprises headlampcontrol; wherein said control is operable to adjust said image sensor toat least two settings including at least a first setting and a secondsetting; wherein said control is operable to process captured image datavia at least two processing techniques; wherein said control is operableto synchronize said at least two settings of said image sensor and saidat least two processing techniques to extract information from saidcaptured image data for said first and second applications; wherein saidcontrol adjusts a setting of said image sensor to said first setting tocapture image data suitable for said first application; wherein saidcontrol adjusts a setting of said image sensor to said second setting tocapture image data suitable for said second application; wherein saidimage processor processes captured image data captured by said imagesensor at said first setting via a processing technique suitable forsaid first application; wherein said image processor processes capturedimage data captured by said image sensor at said second setting via aprocessing technique suitable for said second application; wherein saidcontrol is operable to process captured image data for a thirdapplication; wherein said imager and at least a part of said control aredisposed in an accessory module of the equipped vehicle and wherein saidaccessory module is mounted at the inner-cabin surface of the windshieldof the equipped vehicle; wherein said driver assistance system comprisesan auxiliary sensor; wherein said auxiliary sensor is selected from thegroup consisting of (i) an ultrasonic sensor, (ii) a radar sensor and(iii) a lidar sensor; wherein said auxiliary sensor is operable todetect an object exterior the equipped vehicle; and wherein, responsiveat least in part to detection of the object by said auxiliary sensor, atleast one of (i) a setting of said image sensor is adjusted and (ii)processing of image data that corresponds to a region at which theobject is detected is enhanced.
 17. The driver assistance system ofclaim 16, wherein said control is operable to at least one of (a)provide enhanced imaging of an object viewed by said imager and (b)provide enhanced processing of at least a portion of an image capturedby said imager that corresponds to a region exterior the equippedvehicle at which an object is viewed by said imager.
 18. The driverassistance system of claim 16, wherein at least one of (a) said controlis operable to selectively or intermittently or occasionally adjust asetting of said image sensor to extract information from said capturedimage data for headlamp control and for lane departure warning and (b)wherein said control is operable to selectively or intermittently oroccasionally adjust a processing technique to extract information fromsaid captured image data for headlamp control and for lane departurewarning.